A Tight Estimate for Decoding Error-Probability of 
LT Codes Using Kovalenko's Rank Distribution 



Ki-Moon Lee Hayder Radha Beom-Jin Kim 

Dept of Info. & Comm. Eng Department of ECE Department of Mathematics 

Sungkyunkwan University Michigan State University Yonsei University 

Suwon Kyunggi, R.O.Korea E. Lansing, MI 48824, USA Seoul, R.O.Korea 

Email: leekimoo@skku.edu Email: radha@egr.msu.edu Email: beomjinkim@yonsei.ac.kr 



Abstract — A new approach for estimating the Decoding Error- 
Probability (DEP) of LT codes with dense rows is derived by using 
the conditional Kovalenko's rank distribution. The estimate by 
the proposed approach is very close to the DEP approximated 
by Gaussian Elimination, and is significantly less complex. As a 
key application, we utilize the estimates for obtaining optimal 
LT codes with dense rows, whose DEP is very close to the 
Kovalenko's Full-Rank Limit within a desired error-bound. 
Experimental evidences which show the viability of the estimates 
are also provided. 

I. Introduction and Backgrounds 

For Binary Erasure Channels (BEC), the task of a Luby 
Transform (LT) decoder is to recover the unique solution of a 
consistent linear system 

HX T = f, / 3 = (/3 1 ,...,/3 m )e(F^) m , (1.1) 

where H is an m x n matrix over F 2 . This can be explained 
briefly as follows. (For detailed backgrounds, see [l]-[3]). 
In LT codes, to communicate an information symbol vector 
a = (a>i,...,a n ) € (F|)", a sender constantly generates 
and transmits a syndrome symbol fii = Hia T over BEC, 
where Hi 6 F£ is generated uniformly at random on the 
fly by using the Robust Soliton Distribution (RSD) p{x) [1]. 
A receiver then acquires a set of pairs {(Hi t , /^J}^ and 
interprets it as system ( II. U . and hence, the variable vector 
X = (xi, . . . , x n ) £ (Fj) 77, represents the information symbol 
vector a. Unlike LDPC codes, the row-dimension rn of H 
is a variable and the column-dimension n is fixed. Thus a 
reception overhead defined as 7 = m ~" is the key parameter 
for measuring error-performance of LT codes. 

System (II. lb has the unique solution X = a iff Rank(i?) = 
n, the full rank of H . In case of the full-rank, a can be re- 
covered by using a Maximum-Likelihood Decoding Algorithm 
(MLDA) such as the ones in [5], [15]. These algorithms are 
an efficient Gaussian Elimination (GE) that fully utilize an 
approximate lower triangulation of H, obtainable by exploiting 
the diagonal extension process with various greedy algorithms 
in [4]-[6], [15]. Under those GE, thus, the probability of de- 
coding success is the full-rank probability Pr(Rank(i?) = n). 

It is shown in [1] by Luby that, when H is generated by 
the RSD with large n and 7 > \n(n/5)/y/n, system ( II. Il l 
can be solved for X = a by using the Message Passing 



Algorithm (MPA) in [7] with the minimum probability 1 — 5, 
and the number of row operations to compute X — a by 
the MPA is 0(n ln(n/<5)). For short n, however, a stable 
overhead 7 needed for successful decoding by the MPA with 
high probability is not trivial. In fact, even under GE that is 
much superior to the MPA in error-performance, a stable 7 to 
achieve the full-rank probability near one is not trivial. 

Let H of system ( II. lb be an m x n binary random matrix 
generated by a row-degree distribution p(x) = Pd% d with 
m = (1+7)71. The Decoding Error Probability (DEP) of an LT 
code generated by p(x) used in this paper is the rank-deficient 
probability defined as 

P£(l + 7,n,p) = 1 - Pr(Rank(ff) = n) (1.2) 
= Pr(Rank(ff) < n). (1.3) 

Then with a desired error-bound 6 € (0, 1), define 

7min(<5, n, p) = min{ 7 | P^(l + 7, n, p) < 5}, (1.4) 

and refer to as the Minimum Stable Overhead (MSO) of the 
code with 5. With m = (1 + 7)71 symbols of where 7 > 
7min(<5, n, p), thus, the recovery of a can be accomplished by 
GE decoding with probability at least 1 — S. 

It was shown in [14] that probabilistic lower-bounds for 
DEP and MSO of random binary codes exist, called Ko- 
valenko's Full-Rank Limit and Overhead (KFRL and KFRO 
respectively). Specifically, KFRL is the function 

tf(l + 7 ,n) = l- jWl-^V k = 1 n, (1.5) 

i=k+i ^ ' 

where K(l + 7,71) < Pg(l + J,n,p). Hence, the DEP of 
LT and LDPC codes cannot be lower than KFRL. Similar to 
MSO, KFRO is the minimum 7 defined as 

j K (5, n) = min{ 7 > | K(l + 7, n) < 6}, (1.6) 

7 

where 7k(<5, n) < 7 m i n (<5, n, p). For successful decoding 
under the constraint Pg^(l + 7, n, p) < 5, thus, the minimum 
number of symbols of j3 that a receiver should acquire is at 
least (1 + 'jK(5,n))n [14, Theorem 2.2]. For short n and 
small 5, since the KFRO Jk(5, n) is not trivial, the MSO 
7min(^, n, p) is not trivial. It was also observed experimentally 



that K(l+j, n) w 2 7 ™ as 7 increases. For small 5, therefore, 
7it(5,n) sa where fc«5 = min{fc e N+ | 2~ fe < <5}. 

Let n(x) = J2d<d ^dX d , a truncated form of RSD where 
linin^oo ™ = 0. For short n, the DEP of codes (generated) by 
a truncated p(x) alone exhibits error-floors over a large range 
of 7. The error- floor region however can be lowered to near 
zero dramatically by supplementing a small fraction of dense 
rows to p{x) (see [12]— [14]). The row-degree distribution p(x) 
considered in this paper is thus a supplementation of p(x) with 
a fraction of rows of degree ^ as shown below 



P{x) = E 



d<d.Q 



(J>d 



^/2 



k > 0. (1.7) 



By rearranging rows, an H by p(x) above can be expressed 



as H 



r , where H is a sparse matrix generated by fi(x) 

and H is a dense one formed by random rows of degree 
The key objective of optimizing LT codes in this paper is to 
obtain a p(x), by which, generated LT codes can achieve the 
7min(#, n, p) near the jk(S,ti) for better error-performance, 
but the dense fraction p n / 2 = y^r is as small as possible for 
encoding and decoding efficiency. 

In the paper [15], a simple way of using an Upper-Bound of 
DEP (UBDEP) was formulated for the fast optimization. This 
approach was quite effective in that, an estimate of the UBDEP 
by the formulation is close to the DEP approximated by GE 
and is obtainable very rapidly (within a fraction of a second 
using a standard computer). Hence, the optimization was 
accomplished very rapidly as well by checking the estimates 
with various fractions for p n /2- 

In this paper, an exact formulation of DEP is derived by de- 
composing the full-rank probability in (II. 2b as a sum of condi- 
tional full-rank probabilities, that are computable quite rapidly 
by using by the conditional Kovalenko's rank-distribution. The 
formulation is similar to that of UBDEP in that, it uses prior 
knowledges of the rank-distribution of the sparse part H. 
The Estimate of DEP (EDEP) by the formulation is however 
extremely close to the DEP approximated by GE, and also is 
computable quite rapidly (again, within a fraction of a second 
on a standard computer). Thus, a finer optimization of p{x) can 
be accomplished very fast by checking EDEP's with various 
fractions for p n /2- 

The remainder of this paper is organized as follows. In 
Section [Til a simple approach for generating a truncated 
RSD for a supplementation p{x) in (1I.71 > is presented. In 
Section [III] explicit formulations of DEP and UBDEP are 
derived by analyzing the full-rank probability in dl.21 > and the 
rank-deficient probability in ( II. 3b . respectively, and are utilized 
for obtaining an optimal p(x). The KFRL in ( II. 5b is also 
explained by the conditional Kovalenko's rank-distribution in 
this section. In Section IIVI further experimental results which 
show the viability of the EDEP are presented. This paper is 
summarized in Section [V] 



II. A Truncation of an RSD 

The RSD considered for the truncation in dl.7l ) is the one 
in [3, Ex. 50. 2]. Let h(d) denote the expected number of rows 
of degree d of an H by p(x). With S > 1, we have 



h{d) 



'S+l, 



K dJd-Tj 

Setting the number of rows of H as 



if d=l 
4 , otherwise 



m = h ( d ) ~n + S(l + ln(n)), 



i=l 



then normalizing h(d) by the m yields 



where 



Mc2 



d=l 



h(d) 



m 



(II. 1) 



(II.2) 



(II.3) 



For more detail, see [15, Section-IV]. 

Consider now a truncated RSD in such a way that p(x) = 
J2d<d ^d.x d , where Hindoo ^ = 0. Let H be now anmxn 
matrix generated by a truncated p(x). The reason behind this 
truncation is that in practice, most of the fractions of the p(x) 
in ( III. 3b are too small to get mpd > 1- Fractions for higher 
degrees however should be assigned appropriately to meet the 
constraint on the density (in number of nonzero entries of H), 



a r = y^d-fj,d 

d<dn 



> 



ln(n/e) 
(1 + 7)' 



(II.4) 



where a r is the average row-degree of H. By doing so, all 
columns of H of system (II. j} are nonzero with probability at 
least 1 — e. This constraint can be explained by looking at the 
column-degree distribution of H, 



\(x) = n 



d=l 



1 



(l+-f)nfj, d 



(11.5) 



as follows. From X(x), we have n\(0) ps ne~( 1+7 ) Qr as the 
expected number of null columns of H. With an appropriate 
do, therefore, fractions of p(x) should be assigned to meet the 
inequality ne~( 1+7 ) ar < e < 1, that is equivalent to ill. 4-1 . 

For short n, even if a truncated p(x) meets the constraint 
(III. 4b . experiments exhibited that the DEP of codes by p(x) 
alone exhibits error-floors over a large range of 7. A desirable 
feature of p,{x) is however that, for 7 > 0, the DEP is mainly 
contributed by the rank-deficient probabilities Pr(Rank(i/) = 
n — rf) of small 77. In Fig. Mill for example, the curve Pf£.(l + 
7, 100, /j,), where p(x) is a truncated one in TABLE Mill is 
the DEP approximated by the GE in [15] called the Separated 
MLDA (S-MLDA). As can be seen clearly, it never reaches 
the bound 5 = 10~ 3 for 1 + 7 < 1.3. As 7 increases, on the 
other hand, it is very close to the deficiency curve 77 = 1 and 
is almost identical to the sum of deficiency probabilities of 
i] = I, ... ,7. These deficiency probabilities can be lowered 
to near zero dramatically when enough number of dense rows 
are supplemented to H. The portion of the density increased 
by the dense rows alone however could be much larger than 



expected. Therefore, with a desired 6, the fraction for p n i 2 of 
p(x) in ( II. 7t should be as small as possible, while maintaining 
the -f m in{S,n,p) near the j K (8,n). 

Let 2?i = {1, 2, . . . , d s }, and let V 2 be a set of few spike 
degrees d such that d s < d < do for some do- A truncation 
of p(x) is summarized as follows. 

Rl) Generate the p(x) in (111.31 with desired S 1 , n, and e. 

R2) Take a few spike terms for T) 2 , if necessary, such that 

J2dev 2 = 1 - Edexh M> and at me same time to 
hold a r > as in dfab . 

Thus, hopefully, columns of by a truncated /i(+) have a 
one in some rows of degree d € X>i U 2?a with probability 
at least 1 — e. An exemplary generated by Rl and R2 
is listed in TABLE IIII.ll and its supplementation p(x) with 
various fractions for p n i 2 was used for computer simulations 
presented in Fig. IIII.2I and Fig. II V. 1 1 

III. The Proposed Estimate for DEP of LT Codes 

We first introduce the optimization of p(x) in [15] that uses 
estimated UBDEP's. Let H of system dl.lt be generated by a 
supplementation p(x) in (II. 7t with m = (1+7)71. By rearrang- 
ing rows, we have H — 5 , where the sparse part i7 is gen- 
ri 

erated by a truncated p(x) and the dense part H is formed by 
random rows of degree 2 Let _B(m, fc,p) = (™)p fe (l— p) m ~ k 
be the Bernouli probability with < p < 1. Assume that the 
dense part attains k rows with probability B(m, k, p n / 2 ). 
With the sparse part i/, let (pu,{rj) = Pr(Rank(£f) = n — rj). 
Let & p (k,rj) = Pr(Rank(iJ) < n\k,rj) the conditional prob- 
ability that Rank(if) < n, given that H attained k rows and 
Rank(ff) =71-77. Finally, let <? p (fc) = Pr(Rank(#) < n|fc) 
the conditional probability that Rank(iJ) < n given that H 
attained k rows. We have i9 p (fc) = X^=i ^pi^iV) ' P^iv)- 

The UBDEP in [15] was formulated in two steps: first by 
expressing the DEP in dl.3t as the sum 



err 



(1 + 7, n, P ) = J2 B ( m ' fc ' ' ^p( fc )' CHI.1) 



fc=0 



second, by finding an upper-bound for $ p (k) as shown in the 
following theorem. (For the proof, see [15, Theorem IV]). 

Theorem HL1 (The UBDEP of LT codes by p(x)). Since 
, &p(k,rj) < f or 1 — V — k by the union bound and 

"& p {k, 77) = 1 for r] > k, we have 

< f *M = E §§r + E I • ( in - 2 ) 

\ »7=1 r;>fc / 

77i/i yields the UBDEP as shown below 

m 

Pf rr {l + 7, «, P) < E B ( m ' fc ' <°«/2) ' ^W- CHI.3) 

fc=0 

Notice that, once the deficient probabilities 4>n{rj), 77 = 
1,2,..., 770, are estimated for some 770 (e.g., the deficiency 
curves r\ = 1, 2, . . . , 27 in Fig. lIlLlT l, the UBDEP in dlTOT l can 
be estimated very fast for any fraction for p n / 2 . Furthermore, 



experiments exhibited that the estimate is also close to the 
DEP approximated by GE over system (ll.lt . Thus the overall 
shape of DEP including its error-floor region is predictable 
from the estimates right away. Exemplary optimizations using 
these estimates are presented in [15]. 

We shall now decompose the Pr(Rank(i7) = 71) in ( II. 2t 
as a sum of conditional full-rank probabilities of H. Let us 
clarify some notations first. With < u> < min{fc, 77}, let 

C(r>, to, k) = Pr(Rank( J ff) = ( n -rj)+ w|jy, k), (III.4) 

the conditional probability that Rank(iZ) = (71 — 77) +ui given 
that Rank(iJ) =71 — 77 an d H attained k rows. Assume that the 
dense part H attains k rows with probability B(m, k, p n / 2 ). 
Let Pr(Rank(i7) = n\k) denote the conditional full-rank 
probability given that H attained k rows. We have, first, 

fe 

Pr(Rank(iJ) = n\k) = ^ C(v, V, k)Mv), (HL5) 
77=0 

where ^(77) = Pr(Rank(_ff) = 77 — 77). Second, 

m 

Pr(Rank(ff) = 77) = B(m, k, p n/2 ) 

■Pr(Rank(#) = n\k). (HI.6) 

Then finally, we have 

p£(i+7,»,p) = i-CE9. (111.7) 



An explicit formulation of £(77, w,fe) in (lIII.5t is possible 
by interpreting Kovalenko's rank-distribution [8] — [ 11], [13] as 
the conditional one as shown in the following lemma. 

Lemma III.l (The Conditional Kovalenko's Rank-Distribu- 
tion). For any (rj,w,k) with to < min{?7, k}, we have 



C(r),uj,k) 



S(u,l) 



n 



1 



i—l 



where, with I = k — to, 

w = EE- E ^ 



i 1 =0i 2 =ii ii=ii-i 

holding the recursion 

S{lu, I) = -rSYw -1,0 + S(u, i - 1). 
2' 

Proof: The proof can be accomplished inductively by 
using the recursions (IHI.lOt and 

1 



(III.8) 



(III.9) 



(III. 10) 



C^^fe + l) = C(»7.w-l,fc)(l 
+ ((V,^,k) 



2>i- 



(III. 11) 



For detailed proof, we refer readers to [13, Lemma IV.3]. 



Theorem III.2. Assume that ^(77) in MIL 51 ) are explicitly 
known over a deficiency range 1 < 77 < 770 /or some 7/0 77zen 
the DEP in MIL 71 ) is explicitly computable. 



n 



2 



(A*d)d=i = (0.014,0.481,0.152,0.082,0.048) 
{HdYdLe = (0.034,0.024,0.024,0.012,0.012) 



/i 25 = 0.059, /i 35 = 0.058 



TABLE III.l 

Fractions of p,(x) generated by the steps R 1 ) and R2) with 
S = 10 AND n = 10 3 . 
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1 1.1 
1+y: 1+Overhead 
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1.3 



Fig. III.l. The curves r) = 1,2,..., represent M (r)) = Pr(Rank(i?) = 
n — Tj) approximated by the S-MLDA, where H is generated by the fJ.(x) in 
TAB Le ITTlTI with n = 100. 



Proof: Since ((i],oj, k) in (1III.41 > is explicitly computable 
by the recursions dill. 10b and ( IIII.11I ). (^(77,77, fc) in ( 1111.5b is 
computable by S(rj, fc — 77) J^[^ =:l (1 — 2 — *■). Therefore, the DEP 
in jIH.71 ) is explicitly computable. ■ 
The DEP in ( 1111.7b is very practical in two respects. First, for 
any fraction for p n /2, experiments exhibited that the EDEP is 
almost identical to the DEP approximated by GE. Second, the 
full-rank probability in ( IIII.6I I can be estimated very rapidly, 
and therefore, a fine optimization of p(x) is obtainable very 
fast by checking EDEP's with various fractions for p n n. 
The following example shows the viability that the EDEP is 
very close to the DEP approximated by the S-MLDA over 
system ( II. lb . An exemplary optimization of p(x) using EDEP's 
is also presented in the example. 



Example III.l. In Fig. Mill the deficiency curves 77 = 
1, 2, 3, . . . , 27, represent 4>^{r}) = Pr(Rank(i/) = 77,-77) 
approximated by the S-MLDA over system dllb . In Fig. IIII.2I 
blue curves are DEP's approximated by the S-MLDA, where 
p(x) is a supplementation of the p(x) in TABLE UlI II with a 
dense fraction in p n / 2 = {0.05, 0.10, 0.15}, and dashed curves 
in red are EDEP's of codes by p(x) of having a dense fraction 
in Pn/2 = {0-025, 0.0 5,..., 0.175, 0.20} computed by using 
a truncated version of (MI.6I 1. 

2+5 

dITL6l > := B(m,k,p n/2 )Fr(Rahk(H) =n\k), (111.12) 

k=l-5 

where I = L^^t! • Notice in Fig. Ml. 21 that, for each assigned 



B— KFRL(1+y,100) 

o 2"' ,0 ° 

*-p£(in.ioo,p) 

- -EDEP 




1.1 1.15 1.2 

1 +y: 1 +0 ve r head 



Fig. III. 2. In the figure, p(x) is a supplementation of the p.(x) in TABLe IIIlT1 
with an assigned dense fraction for p n /2 - F° r eacn fraction for p n /2, the DEP 
curve is approximated by the S-MLDA and the EDEP is computed by using 
the finite version in j III. 12) . 



fraction for p n /2, the EDEP by (Mil 2b above is almost 
identical to the blue one approximated by the S-MLDA. 

Let 6 — 10~ 4 be a given error-bound. Notice from the graph 
of ivT(l+7, 100) that 7 X (10" 4 , 100) := 0.14. Assume that we 
want a p(x) such that 0.14 < 7 min (10~ 4 , p, 100) < 0.15. By 
checking EDEP's with various fractions for p n /2, we see that 
the dense fraction should be larger than 0.125, but the fraction 
Pn/2 = 0.15 is large enough for the optimal p(x). With 5 = 
10" 6 and -y K (l0- 6 , 100) = 0.2, similarly, we get p n/2 « 0.20 
for the constraint 0.2 < 7 min (10~ 6 , p, 100) < 0.21. ■ 

The KFRL in [14] can be explained by a particular case of 
C(?7, to, k) in fllOT l. To see this, let H = H so that H = 0. By 
replacing the k with 71 + k, 77 with n, and ui with 77, — s (hence 
I = k + s) in ( IIII.8b . we have a finite version of Kovalenko's 
rank distribution with q = 2 as shown below 



Pr(Rank(i?) 



S(n 



2(fe+s)s 



Since linin^oo S(n — s,k + s) 
sequence {S(n — s,k + s)}^ =s 



■prfc+S 



n 

i=s+l 



1 

(III. 13) 
and the 



Pr(Rank(7?) 



s) < 



, s is increasing, we have 

1 nr= s+ i(i-F 



2s(k+s) 



nk+s 
i=l 



1-i 



(III. 14) 



The KFRL in (II. 5b is then a particular case of the upper-bound 
above with s = 0. Observe in Fig. IIII.21 that as p n / 2 increases 
the DEP approaches closer to the limit K(l + 7, 100). Notice 
that the KFRL is almost identical to 2 *" 100 as 7 increases. 

IV. Further Experimental Results 

In this section, experimental results which show the viability 
of the EDEP for other block-lengths are presented. The 
truncated RSD in TABLE Mill was used for the experiments 
with the block lengths n = 200, 300, 400, 500. 



P£(1+Y.n,P> 
KFRL(1+v,n) 
EDEP 




(n=200.p =0.10,8=1(0 
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1+y: 1+Overhead 



Fig. IV. 1. Curves in the figure represent EDEP (dashed in red) by 4III.7t . 
DEP (blue) by the S-MLDA, and KFRL (red) with destined error-bounds 
S = 1(T 4 for n = 200, 10~ 5 for n = 300, 400, and 10" 6 for n = 500. 
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Fig. IV.2. Curves represent the fractions e = £ with n = 200, 300, 400, 500 
by the diagonal extension process over an H by p(x) used in Fig. IIV.2I 



By using EDEP's with various fractions for p n /2, we first 
investigated triple pairs of {n,p n / 2 ,S) for an optimal p{x) in 
advance as shown in Fig. II V. II where 5 is a destined error- 
bound. With (500,0.055, 10 -6 ), for example, the fraction 
P250 = 0.055 is large enough for the supplementation p(x), 
achieving the MSO 7 min (10~ 6 , 500, p) that is near the KFRO 
7_r-(10~ 6 , 500) « 0.04. Then with each optimized p(x), 
we approximated the DEP by applying the S-MLDA over 
system ( II. lb . As can be seen clearly, each EDEP by p(x) is 
almost identical to the DEP approximated by the S-MLDA, 
and also, it is very close to the limit K (1 + 7, n) up to a 
destined error-bound 5. 

Let us now discuss the efficiency of LT decoding under the 
S-MLDA. To solve system Oh like other MLDAs in [4]- 
[6], the S-MLDA uses an approximate lower-triangulation of 
H in such a way that H = PHQ T = [£ g], where (P, Q) 
is a pair of row and column permutations obtainable by the 
diagonal extension process in [4], [5], [15], and the right-top 
block B is an I x I lower triangular matrix with I = n — r close 
to n. The successful decoding by MPA in [7] is a particular 
= 0. If [£] ^ then the S-MLDA transforms 
by pivoting columns of the right-block [p] 



case when [ ^ 



H to 



A /ixi 
C 



from the first to the last column of it, and then transforms the 
r x (r + jn) block C to [ Ir * T ] by a conventional GE. 

Let r — en. Since C is not sparse in general, the decoding 
complexity of the transformation by the GE is 0((7e 2 +e 3 )n 3 ). 
Hence the overall complexity of decoding by the MLDA's is 
dominated by either 0((^e 2 + e 3 )n 3 ) or the density \H\. Thus, 
although its overall complexity is 0(n 3 ), the efficiency of the 
LT decoding under the S-MLDA can be measured in terms of 
the fraction e = — , and this is particularly true for short n. 

In Fig. IIV.2I curves represent the fraction e = — obtained 
by the diagonal extension process on H generated by the p(x) 
used in Fig. IIV.ll When n = 500 and 1+7 = 1.1, for instance, 
the point (1.1, 0.038) indicates that, with a 550 x 500 random 
H by the p(x) with p„/ 2 = 0.055, the column-dimension of C 
is about r = 20 that is much smaller than n = 500 the column- 
dimension of H. This substantiates that decoding of the codes 
under the S-MLDA becomes very efficient as 7 increases. 

V. Summary 

In Section [II] a simple approach of generating a truncated 
RSD is presented. In Section [Till explicit formulations of 
DEP and UBDEP are derived and utilized for obtaining an 
optimal p(x), and KFRL is explained as a particular case of 
the conditional Kovalenko's rank-distribution. In Section [IV] 
experimental results which show the viability of the EDEP are 
presented. 
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